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ABSTRACT 



Effective staff development for educators is ongoing and 



intensive; consequently, school districts are heeding this research and 
engaging in long-term contracts with technical assistance providers. Rural 
districts need external support provided by partnerships and need such 
assistance to meet state and community performance expectations. The federal 
Comprehensive School Reform Demonstration Program encourages schools to adopt 
an effective school reform model developed by an external organization. As 
ongoing staff development becomes more widespread and typical of what 
districts and schools engage in, an important question is what evaluation of 
these partnerships should look like. This paper discusses current research 
regarding collaborative partnerships and presents the types of partnerships 
that school districts have reported in a research study being conducted by 
the Regional Educational Laboratories. Four stages of partnership occur in a 
sequential progression toward collaboration: connection, cooperation, 
coordination, and collaboration. Important issues regarding evaluation of the 
various partnership types are addressed. (DFR) 
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Addressing Evaluation of Collaborations: 

Capacity Building by School Districts 

Judy E. Florian, Ph.D. 

Mid-continent Research for Education and Learning 

Research has found that effective staff development for educators is ongoing and intensive (U.S. 
Department of Education, 1999). Districts are heeding this research, and engaging in long term 
contracts with technical assistance providers. In particular, districts in rural settings are in need 
of the external support that partnerships can provide, and many districts are confronted with the 
need for external assistance in order to meet state and community performance expectations. 

An example of policy supporting externally provided staff development within a school is the 
federal Comprehensive School Reform Demonstration Program (CSRD), in which schools are 
strongly encouraged to adopt an effective school reform model as developed by an external 
organization. As ongoing staff development becomes more widespread and typical of what 
districts and schools are engaging in, an important question is what evaluation of these 
partnerships should look like. 

In this paper, current research regarding collaborative partnerships is presented. If a school or 
district is to engage in ongoing professional development, the relationship between the school 
and the external service provider will likely conform to one of the forms of partnership described 
here. 

Next, the types of partnerships that school districts have reported in a research study being 
conducted by the Regional Educational Laboratories as effectively promoting education reform 
are presented. The importance of ongoing, collaborative relationships with technical assistance 
organizations that support the capacity of the district to conduct evaluation and make data-driven 
decisions emerges as important in these examples. Specifically, once partnerships have ended, 
the district needs to have acquired the ability to make data-based decisions about effective 
policies and practices, in particular because the context in which schools and districts operate is 
currently continually changing as states adopt new policies regarding education reform. 

Lastly, important issues regarding evaluation of the various partnership types is also addressed in 
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this paper. Considering that effective partnerships between school districts and external 
organizations were found to be coordinative or collaborative in nature, evaluation of these 
partnerships will also need to be partly or wholey collaborative. Key elements of evaluations of 
by type of partnership are presented. 

Interorganizational collaboration 

Recent research has contributed to our understanding of what makes partnerships successful. For 
the purposes of this paper, any relationship between organizations will be considered a 
partnership. Reading the materials of a particular organization on a widespread basis, attending a 
conference or training session, and contracting for short- or long-term professional development 
are all considered examples of partnerships. According to one model, four stages of partnership 
occur in a sequential progression toward collaboration: Connection, Cooperation, Coordination, 
and Collaboration (Intriligator, 1994; Krueger, 1996). A brief description of these four phases 
follows. 

Connection The first stage is Connection among partners. Partners use the connection 
stage to communicate and gather information. There may or may not be a leader at this stage, but 
generally there is an initiator who desires the connection and sharing of information. 

Cooperation In the second stage, Cooperation among partners takes place. Generally, an 
agreed-upon activity or task is undertaken by the partners at this stage, and as a result 
communication among partners increases. The dimensions of communication include frequency, 
intensity, and amount and level of information exchanged (Krueger, 1996). Increased 
communication produces changes in (and if successful, increases in) responsibility and trust 
among partners. There may or may not be a leader of the partnership at this stage, and the 
activities of the partnership tend to be ones partners could each complete on their own. 

Coordination In the Coordination stage, partners are aligned around common goals or 
objectives and work together to achieve those objectives. At this stage, the partners are necessary 
constituents of the joint work because projects tend to be ones that partners could not complete 
on their own. The projects and activities become the focus of the partnership, and leadership is 
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shared among participants. Participants might adopt specific roles depending on the resources 
and skills they bring to the partnership. 

Collaboration The model’s final stage is Collaboration, which includes many elements of 
coordination, with the addition of long-term contact and joint work, and long-range goals 
promoting continued work together. Communication is deep and broad, trust is strong, and time 
and effort is devoted to the partnership. The partnership becomes dynamic and continues to 
change through more intense contact and joint work. Partners contribute resources to a 
collaboration, and partners desire to have the collaboration meet their individual interests and 
needs as well as everyone else’s needs. 

Mattessich and Monsey (1992) analyzed twelve research studies of collaboration and identified 
important partnership elements. In their paper, Mattessich and Monsey describe the development 
of collaborations over time in four areas: the vision and relationships grow stronger, the structure 
becomes more formal, leadership and control become shared, and resources are secured. (See the 
Appendix for a complete list of the 1 8 elements of a successful collaboration as identified by 
Mattessich and Monsey.) 

Another recent model of collaboration comes from Himmelman (1996). This model is very 
similar to the ones already described here (i.e., consists of the same four stages enumerated by 
Intriligator and Krueger), but with different partnership areas identified. The areas unique to 
Himmelman’s model include Risk, Resource sharing, and Commitment. Specifically, 
Himmelman suggests that collaboration tends to include higher risk than does networking or 
coordination. Also, resources are likely to be provided by all members of a collaboration, or 
planned for and pursued by the collaborative group. Lastly, commitment to a collaboration tends 
to be ongoing and long term, whereas at early stages in a partnership the commitment is short 
term, and centered around specific activities or tasks. 

Barnett, Hall, Berg, and Camarena (1999) distinguish partnership developmental stages from 
partnership types. Two types most often engaged in by school districts are Barnett et al.’s 
Vendor Model and Collaboration Model. In the vendor model, money or other resources are 
exchanged for services between the organizations. In this type of partnership, the exchange is 
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usually narrow in scope, agreed upon at the outset, and in place for a short period of time. In the 
collaborative model, positive relationships among members of the partnership from all 
organizations becomes vital, and the “purposes and desired outcomes are likely to be less specific 
and more ambitions, which means that outcomes will be less tangible and will be accomplished 
more slowly.” Barnett et al. write that the collaborative model partnership can have substantial 
rewards for the organizations involved, but that these benefits are often delayed and difficult to 
document. 

In conclusion, collaborations develop in a sequence of stages and forms, and this variation should 
be taken into account in evaluation of an ongoing partnership. It is likely that if evaluation of the 
collaboration occurs at all, it will be conducted by a party involved in the collaboration (i.e., the 
external agency and/or the school district) rather than a third party. Next, I present the specific 
partnerships mentioned in a research project the Regional Educational Laboratories are 
conducting regarding 16 school districts undergoing education reform. 

School district collaborations 

The examples of effective collaborations that are presented here come from a research project 
that McREL is engaged in with eight other Regional Educational Laboratories. In this project, 
we are examining 16 selected school districts engaged in education reform within various state 
contexts in order to examine what contributes to success in an increasingly accountability-based 
environment. These 16 districts were identified through solicited recommendations of school 
districts based on three criteria: I) increased student achievement over time, 2) challenged in 
some way (e.g., student diversity, a rural setting, a history of low performance, etc.), and 3) 
heterogeneity of state contexts and geographic setting (i.e., urban, suburban, or small town/rural). 
These districts are not representative of districts in general, but they were chosen to examine 
based on their success at implementing education reform. By identifying the partnerships with 
external agencies that promoted reform as reported by these districts, it is hoped that the forms of 
promising collaborations with school districts can be identified. In these case studies, 
partnerships with external agencies emerged as an important contributor to district success, and it 
is the form and content of those partnerships that will be reported here. 
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Out of a total of 16 districts in the study, 10 districts reported that partnerships with external 
organizations contributed to education reform in their district. Partnerships developed in one of 
three ways: the content is identified in advance by the external agency, the content is mutually 
defined (in only one case), or the content area is identified by the school district. In many cases, 
the partnership was defined in advance, and served both the external agency (in its research and 
development effort) and the school district (in its capacity enhancement effort). For example, 
partnerships with Regional Educational Laboratories emerged as an important reform element in 
some districts, and the content of the collaboration was often defined in advance of the 
partnership. That is, the Regional Educational Laboratories had already identified a project to 
work on, and solicited districts to engage in developing materials with the agency. In another 
case, the state had contracted with the service provider for staff development, and therefore the 
content of that assistance was defined in advance. The service provider in these two examples 
was addressing school district needs at a general level when defining the content of the 
partnership, and the partnership required the external organization to adapt their service to the 
school district’s particular setting. 

In the second type of partnership engagement, the external organization (SERVE) and the district 
are engaged in an ongoing capacity enhancing partnership. In this case, the focus of the 
partnership changes with time, as new challenges and goals for the district emerge. In the third 
type of partnership engagement, the school district identifies a need and pursues a contract or 
partnership with an external agency. Partnerships with national agencies reportedly proceeded in 
this manner (e.g., the National Center for the Economy and Education or NCEE, and the League 
of Effective Schools). 

As mentioned above, 10 districts reported partnerships with external agencies that had 
contributed to their education reform effort. In some cases, these partnerships had begun 
between five and ten years ago, but the district’s current capacity was attributed in part to the 
collaboration. Three of the districts reported more than one partnership that they had engaged in, 
either sequentially or simultaneously, and a total of fourteen partnerships were reported. All but 
one partnership engaged the entire district or a sizeable number of schools in the district. (One 
district had only one school participating in the Comer project.) Table 1 presents the content of 



these partnerships as described by the districts involved. 



Table 1. Content of effective partnerships reported by school districts. 



Main Content of Partnership 


Number of 
Districts 
Reported 


External Organizations 


Assessment and data-driven 
decision making 


6 


Missouri State University, League of 
Effective Schools, NWREL, CREATE, 
NCEE 


Curriculum and instruction 


4 


Learning Literacy Network, McREL, 
PREL, CONnect 


Classroom performance assessment 


3 


McREL, Oregon ASCD, SERVE 


School wide reform 


1 


Comer 



Most of the partnerships described included ongoing interaction between representatives from the 
school districts and the external agency. 1 When the four stages of partnerships are considered, 
these relationships between districts and organizations have progressed beyond communication 
and cooperation to the coordination and collaboration stages. It is likely that the collaborative 
aspect of these partnerships contributed to their impact on the districts in this study (U.S. 
Department of Education, 1999). This finding suggests that a collaborative, ongoing partnership 
is more likely to enhance district capacity than is a partnership consisting of, for example, a 
contract for one day of professional development. 

It is interesting that a majority of the partnerships described involved capacity building of district 
and school level assessment and data-driven decision making. And these partnerships that 
contributed to school and district capacities to collect, analyze and interpret data when making 
decisions. 



'In only one partnership (with the Literacy Learning Network), lead teachers from seven 
of the district’s schools received training that those individuals brought back to the school and 
their colleagues. 
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Implications for evaluation 



As described above in the stage model of collaboration, the goals of a highly collaborative 
endeavor are supported by all participants. If the goals are collaboratively identified, it makes 
sense that the evaluation should be based on one coherent set of indicators. In reality, external 
organizations are likely to be interested in evaluative information that overlaps with the data that 
is useful for the school district. For example, the external agency might wish to evaluate its 
presentation of material to teachers, and this will be of less interest to the school or district. 
However, there are indicators such as student achievement and school climate, depending on the 
basis of the partnership, that will be of interest to both organizations. Evaluating the partnership 
collaboratively includes negotiating to define indicators, create data collection instruments, 
construct timelines, and assess the dynamic internal and external contexts of the school or 
district. 



Table 2. Evaluation foci important for various forms of interorganizational partnerships. 



Partnership 

Form 


Key Partnership 
Elements 


Evaluation Foci ' 


Communicative 

and 

Cooperative 


Independent organizations 
engage in short term 
projects that are not 
considered essential to 
either organization’s 
functioning. Existing 
budgets are used. 


For each individual organization: 

■ Process, and 

■ Outcomes of short term projects 
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Coordinative 


Partners engage in 


For each individual organization: 




projects that each 
organization could not 
complete on its own. 


■ Work towards accomplishing individual 
organizational goals. 




Interagency units with 


Evaluation of the partnership assessing: 




specific functions might 
be established. 


■ Management and leadership, and 




Leadership and 
communication become 
more important. (E.g., an 
ongoing vendor 
relationship) 


■ Process. 


Collaborative 


Sustained relationships, a 


Evaluation of partnership assessing: 




shared vision, and long 
term goals are established. 
Leadership is shared, and 


■ Relationships (including communication 
and trust), 




communication becomes 


■ Identification of mutual goals, 




frequent. Resources are 
attended to by all partners. 
(E.g., an ongoing R&D 
relationship between a 


■ Progress toward accomplishing shared goals 
(including work by subunits of partnership), 
and 




district and organization) 


■ Progress toward intermediary goals becomes 
important when goals are very long term. 



Table 2 presents a cursory summary of important evaluation elements for various types of 
partnerships. In the communicative, cooperative, and coordinative types of partnerships, goals of 
individual organizations will remain unique to each agency, and thus evaluations by each 
organization are still necessary. In addition, the more coordinative a partnership becomes, the 
more important management, leadership, and communication are to all partners involved, and 
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these elements can thus be assessed as part of a mutual evaluation of the partnership. 

If a true collaboration is occurring, then the goals of the collaboration are shared, and one 
evaluation of the collaboration should suffice. That is, understanding how and why education 
improvement was or was not promoted should be of interest to both collaborators, and the 
information held by all members of the partnership are necessary for a thorough understanding of 
the collaboration’s progress. 



Partnerships engaged in by education systems can be of varying forms, and these types of 
interorganizational relationships should be taken into account when evaluation is conducted. For 
example, more collaborative partnerships rely more heavily on relationships among organizations 
and members and progress toward long range goals. Thus, evaluation of a true collaboration 
should address these aspects of the partnership. However, evaluation of a cooperative or 
coordinative partnership might focus more on process, effect, and impact of the endeavor. 

In addition to importance of adapting evaluation to the type of partnership engaged in, this 
research addressed the content of partnerships that contribute to educational reform. Districts 
reported that the capacity to evaluate instructional and organizational policies and practices is 
something that can be and often is enhanced through partnerships. Six of the 14 effective 
partnerships reported in this study had data collection and data-driven decision making as a 
major focus. After these partnerships are dissolved or involvement is decreased, the ability of the 
school or district to assess the effects of changes in policy and practice on its own is important. 



Barnett, B.G., G.E. Hall, J.H. Berg, & M.M. Camarena (1999). A framework of various 
partnerships for promoting innovation. Aurora, CO: Mid-continent Research for Education and 
Learning. 

Himmelman, A.T. (1996). On the theory and practice of transformational collaboration: 
Collaboration as a bridge from social service to social justice. In C. Huxham (Ed.), Creating 
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Appendix 

Factors Influencing Success of Collaboration 
From Mattessich and Monsey (1992) 

Environment 

1 . A history of collaboration or cooperation in the community 

2. Collaborative group seen as a leader in the community 

3. Political/social climate favorable 
Membership Characteristics 

1 . Mutual respect, understanding, and trust 

2. Appropriate cross-section of members 

3. Members see collaboration as in their self-interest 

4. Ability to compromise 
Process/Structure 

1 . Members share a stake in both process and outcome 

2. Multiple layers of decision-making 

3. Flexibility 

4. Adaptability 

Communication 

1 . Open and frequent communication 

2. Established informal and formal communication links 

Purpose 

1 . Concrete, attainable goals and objectives 

2. Shared vision 

3. Unique purpose 
Resources 

1 . Sufficient funds 

2. Skilled convener 
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